<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html lang="en-GB" xml:lang="en-GB" xmlns="http://www.w3.org/1999/xhtml">
<head>
  <title>DAWG: Test case structure</title>
  <link rel="stylesheet" type="text/css" href="http://www.w3.org/StyleSheets/TR/base.css" />

<style type="text/css">

  pre.diag { background-color: #f0f0f0 ; padding: 0.5em ; border: 1px solid black ; margin-right: 4em ; }
  span.code { font-family: monospace;}
  pre.cvs-id {color: gray}

#tableResult td, th { padding-left:0.5em; padding-right: 0.5em; padding-top:0.2ex ; padding-bottom:0.2ex ;}

/* Tables */
table, td	{ text-align: left; }
table, td, th	{ border-style: solid;
                  border-width: 1px;
                  border-color: black;
                  border-bottom-color: gray;
                  border-right-color: gray; }
table.plain	{ border-spacing: 0px; padding: 0px ; border-collapse: collapse ; }
                  /* cellpadding="0" cellspacing="1" style="border-collapse: collapse */


  </style>
  <meta name="RCS-Id" content="$Id: README.html,v 1.20 2007/11/27 17:33:43 lfeigenb Exp $" />
</head>
<body>
  <div class="head">
    <a href="http://www.w3.org/"><img height="48" width="72" alt="W3C" src="
    http://www.w3.org/Icons/w3c_home" /></a>
<h1 id="main">DAWG: Test case structure</h1>
  <dl>
      <dd>&nbsp;</dd>
      <dt>Document Editor</dt>
      <dd><a href="mailto:swh@ecs.soton.ac.uk">Steve Harris</a> &ndash; 
      IAM Research Group, Southampton</dd>
      <dd><a href="mailto:j.broekstra@tue.nl">Jeen Broekstra</a> &ndash; 
		Information Systems Group, Eindhoven University of Technology</dd>
       <dd><a href="mailto:feigenbl@us.ibm.com">Lee Feigenbaum</a> &ndash;
        IBM</dd>
      <dt>Version:</dt>
      <dd><code>$Revision: 1.20 $</code></dd>
  </dl>

    <p class="copyright"><a href="
    http://www.w3.org/Consortium/Legal/ipr-notice#Copyright">Copyright</a> &copy;
    2004 <a href="http://www.w3.org/"><acronym title="
    World Wide Web Consortium">W3C</acronym></a><sup>&reg;</sup> ( <a href="
    http://www.csail.mit.edu/"><acronym title="
    Massachusetts Institute of Technology">MIT</acronym></a>, <a href="
    http://www.ercim.org/"><acronym title="
    European Research Consortium for Informatics and Mathematics">ERCIM</acronym></a>,
    <a href="http://www.keio.ac.jp/">Keio</a>), All Rights Reserved. W3C
    <a href="
    http://www.w3.org/Consortium/Legal/ipr-notice#Legal_Disclaimer">liability</a>,
    <a href="
    http://www.w3.org/Consortium/Legal/ipr-notice#W3C_Trademarks">trademark</a>,
    and <a href="
    http://www.w3.org/Consortium/Legal/copyright-documents">document use</a>

    rules apply.</p></div>

<hr title="Separator for header" />
<div>
<h2 id="abstract">Abstract</h2>
<p>This document describes the testing process used by the
<a href="http://www.w3.org/2001/sw/DataAccess/tests">RDF Data Access Working 
Group</a>.</p>
</div>
<div>
    <h2><a name="status" id="status">Status of This Document</a></h2>
    <p>Working Document.</p>
</div>
  <hr />
<p>
The
<a href="http://www.w3.org/2001/sw/DataAccess/">RDF Data Access Working 
Group</a> (DAWG) uses a test-driven process.&nbsp; The
<a href="r2">test area</a> is a 
collection of the current test cases of the working group. </p>
  <p>
Tests are divided into collections (corresponding to directories) for manageability.&nbsp; Each collection 
of tests has a <em>manifest file</em> within its directory (usually named
<tt>manifest.ttl</tt>, but sometimes <tt>manifest.n3</tt>).
There is also a pair of overall manifests containing entries pointing to the
individual test collection manifests: One is a <a
href="http://www.w3.org/2001/sw/DataAccess/tests/data-r2/manifest-syntax.ttl">manifest
of syntax-only tests</a> (positive and negative tests); the other is a <a
href="http://www.w3.org/2001/sw/DataAccess/tests/data-r2/manifest-evaluation.ttl">manifest
of query-evaluation tests</a>.</p>

<h4>Reorganization</h4>
<p>
The Working Group recently completed reorganizing the test suite. The
output of this reorganization process can be found in the <a
href="data-r2/">data-r2</a> directory. The tests here are restructured
copies of tests from the previous test suite along with new tests aimed at
covering as much of the SPARQL Query Language as possible. The purpose of this
restructuring is to enhance usability, clear away obsolete tests and
provide an up-to-date, consistent, and easy-to-use suite of test cases that
SPARQL query language implementors can use to evaluate and report on their
implementation.
</p>
<p>
The Working Group decided on <a
href="http://lists.w3.org/Archives/Public/public-rdf-dawg/2007JulSep/att-0096/21-dawg-minutes.html#item05">21
Aug 2007</a> that the tests as-is constitute a test suite that the group will
use to generate an implementation report for the SPARQL Query Language for
RDF.</p>
</p>
  <h4>Manifest Vocabularies</h4>

  <p>The DAWG's test manifest files define three vocabularies to express
  tests:</p>

  <ol>
    <li><a href="test-manifest.n3">manifest vocabulary</a> (prefixed with
    <tt>mf:</tt> below)</li>
    <li><a href="test-query.n3">query-evaluation test vocabulary</a> (prefixed
    with <tt>qt:</tt> below)</li>
    <li><a href="test-dawg.n3">DAWG test approval vocabulary</a> (prefixed
    with <tt>dawgt:</tt> below)</li>
  </ol>
  
  <p>All examples below use these prefix bindings (specified in <a
  href="http://www.w3.org/2001/sw/DataAccess/df1/">turtle</a>):</p>

  <pre class="diag">@prefix rdf:     &lt;http://www.w3.org/1999/02/22-rdf-syntax-ns#&gt; .
@prefix rdfs:    &lt;http://www.w3.org/2000/01/rdf-schema#&gt; .
@prefix mf:      &lt;http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#&gt; .
@prefix dawgt:   &lt;http://www.w3.org/2001/sw/DataAccess/tests/test-dawg#&gt; .
@prefix qt:      &lt;http://www.w3.org/2001/sw/DataAccess/tests/test-query#&gt; .
  </pre>

  <h4>Manifest Structure</h4>

  <p>
A manifest is a list (RDF Collection) of tests. Every test has 
a name (<tt>mf:name</tt>); many tests also have a comment
(<tt>rdfs:comment</tt>) explaining the purpose of the test. The <tt>dawgt:approval</tt> predicate relates a test
to its official Working Group status (e.g. <tt>dawgt:Approved</tt>). Tests are
grouped (via their <tt>rdf:type</tt>) as syntax tests (an <tt>rdf:type</tt> of
either <tt>mf:PositiveSyntaxTest</tt> or <tt>mf:NegativeSyntaxTest</tt>) or
query-evaluation test (an <tt>rdf:type</tt> of
<tt>mf:QueryEvaluationTest</tt>).</p>

  <h4>Syntax Tests</h4>

  <p>
Each syntax test has an <tt>mf:action</tt>, the object of which is a
resource identifying a (possible) query string. An example definition of a
syntax test is:
  </p>

<pre class="diag">
&lt;#syntax-basic-01&gt;  mf:name  "syntax-basic-01.rq" ;
     rdf:type   mf:PositiveSyntaxTest ;
     mf:action  &lt;syntax-basic-01.rq&gt; ;
     dawgt:approvedBy &lt;http://lists.w3.org/Archives/Public/public-rdf-dawg/2007JanMar/0047&gt; ;
     dawgt:approval dawgt:Approved .
</pre>

<p>
A SPARQL implementation passes a <tt>mf:PositiveSyntaxTest</tt> if it parses
the query string without error. A SPARQL implementation passes a
<tt>mf:NegativeSyntaxTest</tt> if it raises an error while attempting to parse
the query string.
</p>

  <h4>Query-Evaluation Tests</h4>

  <p>Each query-evaluation test has an <tt>mf:action</tt> and an
  <tt>mf:result</tt>. The object of <tt>mf:action</tt> is a resource with
  properties taken from the query-evaluation test vocabulary. At a minimum, a
  test's action includes a <tt>qt:data</tt> relation and a <tt>qt:query</tt>
  relation. The <tt>qt:data</tt> predicate points to a URI
  that can be dereferenced to yield the default graph for the test. The
  <tt>qt:query</tt> prediate points to a URI that can be dereferenced to yield
  the query string for the test. Query-evaluation tests may also use the
  <tt>qt:graphData</tt> predicate to indicate the named graph components of
  the test's RDF dataset.</p>

  <p>Query evaluation tests also contain an <tt>mf:result</tt> which points to
  a URI that can be dereferenced to yield the
  expected results of the test query. These results are expressed in one of
  several possible ways:</p>

  <ul>
    <li>The <a href="http://www.w3.org/TR/rdf-sparql-XMLres/">SPARQL Query
    Results XML Format</a>. SELECT or ASK queries.</li>
    <li>The <a href="result-set.n3">DAWG result-set RDF vocabulary</a>.
    SELECT or ASK queries.</li>
    <li><a href="http://www.w3.org/TR/rdf-syntax-grammar/">RDF/XML</a> or <a
    href="http://www.dajobe.org/2004/01/turtle/">Turtle</a>. CONSTRUCT queries.</li>
  </ul>

<p>A SPARQL implementation passes a query-evaluation test if the graph
produced by evaluating the query against the RDF dataset (and encoding in the
DAWG result set vocabulary, if necessary) is 
href="http://www.w3.org/TR/rdf-concepts/#section-graph-equality">equivalent</a> [RDF-CONCEPTS] 
to the graph named in the result (after encoding in the DAWG result set
vocabulary, if necessary). Equivalence can be tested by 
checking that the graphs are isomorphic and have identical IRI and 
literal nodes. Note that testing whether two result sets are isomorphic is simpler than full graph isomorphism. Iterating over rows in one set, finding a match with the other set, removing this pair, then making sure all rows are accounted for, achieves the same effect.</p>

<p>Query evaluation tests that involve the <a href="http://www.w3.org/TR/rdf-sparql-query#modReduced">REDUCED</a> keyword have slightly different passing criteria. These tests are indicated in the manifest files with the <tt>mf:resultCardinality</tt> predicate with an object of <tt>mf:LaxCardinality</tt>. To pass such a test, the result set produced by a SPARQL implementation must contain each solution in the expected result set at least once and no more than the number of times that the solution occurs in the expected result set. (That is, the expected result set contains the solutions with cardinalities as they would be if the query did not contain REDUCED; to pass the test, an implementation must produce the correct results with cardinalities between one and the cardainlity in the expected result set.)</p>

<p>An example definition of a query-evaluation test is:</p>

<pre class="diag">
&lt;#dawg-regex-002&gt; a mf:QueryEvaluationTest ;
      mf:name    "regex-query-002" ;
      dawgt:approval dawgt:Approved ;
      dawgt:approvedBy &lt;http://lists.w3.org/Archives/Public/public-rdf-dawg/2007AprJun/0029.html&gt; ;
      rdfs:comment
          "Case insensitive unanchored match test" ;
      mf:action
          [ qt:query  &lt;regex-query-002.rq&gt; ;
            qt:data   &lt;regex-data-01.n3&gt; ] ;
      mf:result  &lt;regex-result-002.n3&gt; .
</pre>

  <h4>Test annotations</h4>
  <h5>mf:requires</h5>
  <p>A number of tests in the open-world directory illustrate features of SPARQL 
  by depending on how a SPARQL query processor can extend the set of core types 
  and operations as defined by the operator table [<a href="http://www.w3.org/TR/rdf-sparql-query/">http://www.w3.org/TR/rdf-sparql-query/#OperatorMapping</a>].</p>
  <p>These tests are marked by property <code>mf:requires</code> and an object value from one 
  of the URIs described below.</p>
  <dl>
    <dt><code>mf:XsdDateOperations</code></dt>
    <dd>Requires the processor to understand comparisons of 
    literal of type xsd:date. Without proivding operations on the xsd:date datatype, a 
    processor would raise an error&nbsp;on the operations of &quot;=&quot; and &quot;!=&quot; 
    etc. With 
    an understanding of xsd:date, a processor can perform value-based operations 
    and provide the operations described in &quot;XQuery 1.0 and XPath 2.0 Functions 
    and Operators&quot; (e.g. <a href="http://www.w3.org/TR/xpath-functions/">
    date-equals</a> <a href="http://www.w3.org/TR/xpath-functions/">
    date-less-than</a>)</dd>

	<dt><code>mf:StringSimpleLiteralCmp</code></dt>
    <dd>This indicates that the test uses the fact that plain literals, without 
    language tags test are the same value as an xsd;string with the same 
    lexicial form. This is covered by rules &quot;xsd 1a&quot; and &quot;xsd 1b&quot; from RDF 
    Semantics [<a href="http://www.w3.org/TR/rdf-mt/">http://www.w3.org/TR/rdf-mt/#DtypeRules</a>].</dd>
    
    <dt><code>mf:KnownTypesDefault2Neq</code></dt>
    <dd>This indicates that a processor extends the SPARQL operator model by 
    using the fact that values of literals can be in disjoint value spaces and 
    hence can not be equal by value. For example, an xsd:integer can not be the 
    same value as an xsd:boolean because these two datatypes define disjoint 
    value spaces.</dd>

<dt><code>mf:LangTagAwareness</code></dt>
	<dd>This indicates that the test assumes the SPARQL query processor has support 
for plain literals with language tags.  The minimum set of operators in the 
    <a href="http://www.w3.org/TR/rdf-sparql-query/#OperatorMapping">SPARQL operator table</a> 
does not include language tag handling, only plain literals without language 
tag (simple literals) and certain XSD datatypes.

</dl>

    <h5>mf:notable</h5>
  <p>This annotation indicates a feature of SPARQL that implementers might note:</p>
  <dl>
    <dt><code>mf:IllFormedLiteral</code></dt>
    <dd>The test involves handling of ill-formed literals.</dd>

</dl>

    <p>&nbsp;</p>
  </body>
</html>
